Let’s first load the packages that we need for this chapter.
Data & analysis code was pulled from the ROAR-Score repository on Github (https://github.com/yeatmanlab/ROAR-Score/tree/main/Analysis/SRE_research/data).
We created a list of 200 sentences for Sentence Reading Efficiency, an online, automated assessment in which participants endorse sentences as being true or false. These sentences were created by human researchers who referenced the sentence structure of other sentence reading efficiency tasks such as the Test of Silent Reading Comprehension and Efficiency (TOSREC) and the Woodcock-Johnson Sentence Reading Fluency (WJ-SRF).
167 participants aged 5-29 who were recruited across 3 different studies piloted the task. Based on these data, we conducted item-level analysis to filter out items that are not suitable for the task, leaving us with 136 final sentences.
Setting variables that will be used in code later on
The assessment offers the participants only 2 choices: true or false; participants who are guessing can be expected to get ~50% of the items corrects. Thus, we conservatively remove participants from our sample if they could not correctly endorse more than 60% (0.60) of the items. We filtered out 16 participants, leaving us with 151 participants for the final sample.
df.filtered.participants <- df.resp.matrix.lab.filtered %>%
mutate(keep = 1) %>%
select(subj, keep)We filtered out items that had a proportion correct of 85% or greater for our initial pass of the stimuli bank.
We retained items that had a proportion correct greater than 0.85 (85%) for two reasons: 1) We wanted to ensure that the task would be relatively easy for participants, and 2) we need a sufficient number of sentences so participants are not able to exhaust the whole stimuli pool. We also filtered out items that had 25 or less responses.
After filtering out 38 items, we were left with 162 items that we conducted further analysis on.
quad.plot <- ggplot(data = df.lab.items.filtered.v3, label = sentence, aes(label = sentence)) +
geom_point(data = df.lab.items.filtered.v3, mapping = aes(x = avgRTStd,
y = pCorrStd, color=numCharacter)) +
labs(title = "Response Time vs. Agreement Rate",
color = "Sentence Length \n(Number of \nCharacters)") +
scale_x_continuous(limits = c(-3, 3), breaks = round(seq(-3,3)), "Faster <------- Response Time ----> Slower") +
scale_y_continuous(limits = c(-3, 3), breaks = round(seq(-3,3)),"Less <------- Agreeability ----> More ") +
scale_color_viridis(option = 'viridis') +
theme_light() +
theme(text=element_text(family="Avenir", size=12)) ggplotly(quad.plot)df.lab.items.filtered.v4 <- df.lab.items.filtered.v3 %>%
filter(sentenceId != "lab_56")In order to ensure that we have an equal number of true and false sentences, we used the R package MatchIt to match the items based on item difficulty (or proportion correct). Note that difficulties for each item were defined as:
Easy: Items that have a proportion correct greater than 0.95
Medium: Items that have a proportion correct greater than 0.90 but less than or equal to 0.95
Hard: Items that have a proportion correct less than than 0.90
After matching the easy sentences, 40 sentences (20 true and 20 false) were included in the final set of sentences.
df.easy.excluded sentenceId sentence
1 lab_46 Knives are used for cutting
2 lab_54 Some kids have blue crayons
3 lab_79 Chocolate is a type of candy
4 lab_83 Some people keep birds as pets
5 lab_84 Some people eat pancakes for breakfast
6 lab_135 Breakfast is the first meal of the day
7 lab_138 Some people get to work using the train
8 lab_158 Many people like wearing shoes when they go outside
9 lab_171 When it is raining outside, many fields will become muddy
10 lab_181 Some kids want to become writers when they grow up
answerKey itemNum attempted correct pCorr pCorrStd sum avgRT
1 TRUE 46 31 30 0.9677419 0.8820929 52.38303 1.689775
2 TRUE 54 42 40 0.9523810 0.6655083 129.07389 3.073188
3 TRUE 79 35 34 0.9714286 0.9340733 63.32416 1.862475
4 TRUE 83 45 43 0.9555556 0.7102691 135.12499 3.002778
5 TRUE 84 42 40 0.9523810 0.6655083 181.61277 4.324114
6 TRUE 135 37 36 0.9729730 0.9558488 74.21376 2.005777
7 TRUE 138 35 34 0.9714286 0.9340733 138.04104 3.944030
8 TRUE 158 33 32 0.9696970 0.9096583 128.09033 3.881525
9 TRUE 171 41 39 0.9512195 0.6491324 158.71274 4.069557
10 TRUE 181 37 36 0.9729730 0.9558488 114.99220 3.194228
avgRTStd numCharacter difficulty
1 -1.513551206 23 Easy
2 -0.125084880 23 Easy
3 -1.340220211 23 Easy
4 -0.195752505 25 Easy
5 1.130410378 33 Easy
6 -1.196394600 31 Easy
7 0.748938125 32 Easy
8 0.686204942 43 Easy
9 0.874924119 48 Easy
10 -0.003602929 41 Easydf.lab.items.easy.final sentenceId
1 lab_0
2 lab_1
3 lab_4
4 lab_6
5 lab_8
6 lab_12
7 lab_16
8 lab_25
9 lab_32
10 lab_33
11 lab_35
12 lab_40
13 lab_44
16 lab_55
17 lab_61
18 lab_63
19 lab_69
20 lab_71
21 lab_73
22 lab_74
23 lab_75
24 lab_77
28 lab_85
29 lab_87
30 lab_89
31 lab_90
32 lab_93
33 lab_99
34 lab_106
35 lab_115
36 lab_116
37 lab_122
40 lab_140
42 lab_162
43 lab_164
44 lab_170
46 lab_178
47 lab_180
49 lab_186
50 lab_191
sentence
1 Chairs are alive
2 Fish have thumbs
3 Cars have wheels
4 Trees have branches
5 Sandwiches are food
6 Soda is always brown
7 Children can be sad
8 All cars are yellow
9 Triangles have five sides
10 A monkey has a face
11 People like to eat mud
12 Children enjoy playing with toys
13 Computers need electricity to work
16 Some houses have back yards
17 Some kids take music lessons
18 Spaceships are flown by cowboys
19 A pillow can be very soft
20 Some worms ride pumpkins to school
21 Sweaters can be made of coal
22 Chefs cook meals in their beds
23 Dogs always walk on two legs
24 Trash is thrown away in bathtubs
28 Eyeglasses can help you see better
29 Wolves are a type of monkey
30 It is usually dark at night
31 A blanket can be very soft
32 Towels are often made of glass
33 Most humans listen with their feet
34 Some children are afraid of the dark
35 People can buy tomatoes at the market
36 All children are afraid of the sun
37 Orange juice is always made of grapes
40 The ocean is home to many aquatic animals
42 Trash cans are where you store extra notebook paper
43 Basketball is a game that you play with a fork
44 Some people like to run for exercise in the morning
46 Soccer is a game you play with only your hands
47 A teacher is a person that works at a school
49 Grandparents can bring presents when they come to visit their family
50 When you break a bone you normally have to wear a cast
answerKey itemNum attempted correct pCorr pCorrStd sum avgRT
1 FALSE 0 44 42 0.9545455 0.6960271 91.49515 2.079435
2 FALSE 1 45 45 1.0000000 1.3369207 85.00037 1.888897
3 TRUE 4 42 41 0.9761905 1.0012145 66.08075 1.611726
4 TRUE 6 41 40 0.9756098 0.9930265 67.65853 1.650208
5 TRUE 8 48 47 0.9791667 1.0431778 73.21354 1.525282
6 FALSE 12 43 41 0.9534884 0.6811226 93.69506 2.178955
7 TRUE 16 41 40 0.9756098 0.9930265 66.20243 1.614694
8 FALSE 25 35 35 1.0000000 1.3369207 62.23937 1.778268
9 FALSE 32 40 40 1.0000000 1.3369207 93.43161 2.335790
10 TRUE 33 32 32 1.0000000 1.3369207 51.59473 1.612335
11 FALSE 35 47 45 0.9574468 0.7369352 89.03229 1.894304
12 TRUE 40 37 37 1.0000000 1.3369207 89.03335 2.406307
13 TRUE 44 29 29 1.0000000 1.3369207 75.53671 2.604714
16 TRUE 55 43 43 1.0000000 1.3369207 89.82713 2.089003
17 TRUE 61 39 39 1.0000000 1.3369207 88.94155 2.280552
18 FALSE 63 44 44 1.0000000 1.3369207 127.36084 2.894565
19 TRUE 69 48 47 0.9791667 1.0431778 105.97524 2.207817
20 FALSE 71 34 33 0.9705882 0.9222248 104.39870 3.070550
21 FALSE 73 41 39 0.9512195 0.6491324 125.96161 3.072235
22 FALSE 74 41 40 0.9756098 0.9930265 118.94683 2.973671
23 FALSE 75 42 41 0.9761905 1.0012145 123.97459 2.951776
24 FALSE 77 45 43 0.9555556 0.7102691 136.87046 3.110692
28 TRUE 85 38 37 0.9736842 0.9658770 108.14471 2.922830
29 FALSE 87 41 39 0.9512195 0.6491324 76.64538 1.965266
30 TRUE 89 38 38 1.0000000 1.3369207 85.07677 2.238862
31 TRUE 90 34 34 1.0000000 1.3369207 105.06983 3.090289
32 FALSE 93 43 43 1.0000000 1.3369207 93.89001 2.183489
33 FALSE 99 45 44 0.9777778 1.0235949 116.48522 2.588561
34 TRUE 106 37 37 1.0000000 1.3369207 82.42713 2.227760
35 TRUE 115 37 37 1.0000000 1.3369207 115.53913 3.122679
36 FALSE 116 40 39 0.9750000 0.9844292 96.30437 2.407609
37 FALSE 122 34 34 1.0000000 1.3369207 78.62520 2.312506
40 TRUE 140 38 37 0.9736842 0.9658770 128.94337 3.484956
42 FALSE 162 35 34 0.9714286 0.9340733 197.23792 5.635369
43 FALSE 164 45 45 1.0000000 1.3369207 135.32318 3.007182
44 TRUE 170 51 50 0.9803922 1.0604568 172.20617 3.376592
46 FALSE 178 35 34 0.9714286 0.9340733 122.21326 3.491807
47 TRUE 180 38 37 0.9736842 0.9658770 93.46986 2.459733
49 TRUE 186 38 37 0.9736842 0.9658770 165.01177 4.342415
50 TRUE 191 41 40 0.9756098 0.9930265 166.95486 4.072070
avgRTStd numCharacter difficulty distance weights subclass
1 -1.12246785 14 Easy 0.5585887 1 7
2 -1.31370199 14 Easy 0.6372482 1 3
3 -1.59188587 14 Easy 0.5966747 1 8
4 -1.55326285 17 Easy 0.5956658 1 13
5 -1.67864527 17 Easy 0.6018322 1 18
6 -1.02258443 17 Easy 0.5567070 1 14
7 -1.58890710 16 Easy 0.5956658 1 20
8 -1.42473554 16 Easy 0.6372482 1 2
9 -0.86517627 22 Easy 0.6372482 1 1
10 -1.59127394 15 Easy 0.6372482 1 1
11 -1.30827509 18 Easy 0.5637444 1 20
12 -0.79440217 28 Easy 0.6372482 1 2
13 -0.59527002 30 Easy 0.6372482 1 3
16 -1.11286499 23 Easy 0.6372482 1 4
17 -0.92061586 24 Easy 0.6372482 1 5
18 -0.30436081 27 Easy 0.6372482 1 4
19 -0.99361660 20 Easy 0.6018322 1 6
20 -0.12773249 29 Easy 0.5869091 1 13
21 -0.12604183 23 Easy 0.5526629 1 17
22 -0.22496551 25 Easy 0.5956658 1 15
23 -0.24694053 23 Easy 0.5966747 1 12
24 -0.08744370 27 Easy 0.5603851 1 19
28 -0.27599197 29 Easy 0.5923149 1 7
29 -1.23705399 22 Easy 0.5526629 1 16
30 -0.96245822 22 Easy 0.6372482 1 9
31 -0.10792127 21 Easy 0.6372482 1 10
32 -1.01803426 25 Easy 0.6372482 1 5
33 -0.61148261 29 Easy 0.5994281 1 11
34 -0.97360101 30 Easy 0.6372482 1 11
35 -0.07541288 31 Easy 0.6372482 1 12
36 -0.79309479 28 Easy 0.5946056 1 18
37 -0.88854555 31 Easy 0.6372482 1 9
40 0.28818725 34 Easy 0.5923149 1 14
42 2.44645576 43 Easy 0.5883785 1 8
43 -0.19133219 37 Easy 0.6372482 1 10
44 0.17942712 42 Easy 0.6039494 1 15
46 0.29506376 37 Easy 0.5883785 1 6
47 -0.74078061 35 Easy 0.5923149 1 16
49 1.14877841 58 Easy 0.5923149 1 17
50 0.87744587 43 Easy 0.5956658 1 19df.lab.items.medium <- df.lab.items.filtered.difficulty %>%
filter(difficulty == "Medium")After matching the medium sentences, 64 sentences (32 true and 32 false) were included in the final set of sentences.
df.medium.excluded sentenceId
1 lab_2
2 lab_62
3 lab_72
4 lab_103
5 lab_109
6 lab_132
7 lab_137
8 lab_175
9 lab_197
sentence
1 Cats can drink
2 Squirrels will climb trees often
3 A baker works in the kitchen
4 People can buy chairs in stores
5 Some people put ketchup on hot dogs
6 Some people drink coffee with milk and sugar
7 Wearing sunscreen protects your skin from the sun
8 Many people have the same last name as their parents
9 The president of the United States of America lives in the White House
answerKey itemNum attempted correct pCorr pCorrStd sum avgRT
1 TRUE 2 37 35 0.9459459 0.5747769 81.23219 2.195465
2 TRUE 62 39 37 0.9487179 0.6138612 92.50852 2.372013
3 TRUE 72 36 34 0.9444444 0.5536063 113.64815 3.247090
4 TRUE 103 39 37 0.9487179 0.6138612 105.89910 2.786818
5 TRUE 109 36 34 0.9444444 0.5536063 108.85277 3.110079
6 TRUE 132 35 33 0.9428571 0.5312258 119.19430 3.505715
7 TRUE 137 37 35 0.9459459 0.5747769 118.15913 3.375975
8 TRUE 175 39 37 0.9487179 0.6138612 148.08023 3.896848
9 TRUE 197 38 36 0.9473684 0.5948333 107.93718 2.917221
avgRTStd numCharacter difficulty
1 -1.00601452 12 Medium
2 -0.82882074 28 Medium
3 0.04945240 23 Medium
4 -0.41250052 26 Medium
5 -0.08805895 29 Medium
6 0.30902199 37 Medium
7 0.17880849 42 Medium
8 0.70158407 43 Medium
9 -0.28162165 58 Mediumdf.lab.items.medium.final sentenceId sentence
2 lab_3 Bears can talk
3 lab_7 Mice have ears
4 lab_10 The sun is bright
5 lab_11 Whales have two legs
6 lab_13 All flowers are yellow
7 lab_14 People sometimes drink milk
8 lab_15 Cows sing at night
9 lab_18 Elephants are large animals
10 lab_21 Apples grow on trees
11 lab_22 Kittens are young cats
12 lab_27 Some shoes have laces
13 lab_28 Adults can be sad
14 lab_30 Moms like eating hats
15 lab_31 Many babies drink milk
16 lab_36 Some kids love red crayons
17 lab_37 Lizards like to cook pasta
18 lab_38 Snails are usually very fast
19 lab_39 Pets can keep people company
20 lab_45 Blankets are made of metal
21 lab_49 Every magician is a fairy
22 lab_50 Blankets help keep you cold
23 lab_51 Many nurses work in hospitals
24 lab_53 People like to eat dust
25 lab_58 Tigers sometimes swing at playgrounds
26 lab_60 Some girls play the piano
28 lab_64 Cloudy days are always warm
29 lab_66 People drive cars in the sink
30 lab_70 A curtain is something you wear
32 lab_81 Eating only ice cream is healthy
33 lab_88 People drive cars on the road
34 lab_91 Some slugs ride elephants to school
35 lab_95 Puppies carry backpacks on their back
36 lab_98 Most houses are made of foam
37 lab_100 Many people paint pictures with screwdrivers
38 lab_101 Lollipops are always made from vegetables
40 lab_105 Most houses have grass on the ceiling
41 lab_107 Pockets can be used to keep change
43 lab_110 People brush their teeth with mayonnaise
44 lab_114 A cactus may grow in the desert
45 lab_119 Boys usually wash their hair with yogurt
46 lab_120 Girls can listen to music with earphones
47 lab_121 Climbing up a mountain can be exhausting
48 lab_123 Adults are usually taller than their children
49 lab_124 Dentists always give children candy and lemonade
50 lab_127 A picture can hang on the wall
51 lab_128 People can use cash to pay in stores
53 lab_134 The United States of America is in Africa
54 lab_136 Towels are used to make things dry
56 lab_144 Most girls keep their books in a pond
57 lab_145 Every basketball player is on a school team
58 lab_146 Stores are where you go to buy things
59 lab_153 People can wear socks to keep their feet warm
60 lab_154 Air conditioners can lower the temperature of a room
61 lab_157 Medicine is supposed to make you stay sick longer
62 lab_165 Umbrellas are useful for keeping people dry when it rains
63 lab_166 Spiders like to drink orange juice out of coffee mugs
64 lab_167 Baseball is a game that you play with a knife
65 lab_169 When a traffic light is red you need to stop
67 lab_176 Most dogs wear leashes when they go for a walk
68 lab_179 It is faster to run than it is to walk
69 lab_182 Students never get in trouble when they cheat on tests
70 lab_183 A doctor can work in a hospital that serves many people
71 lab_184 Some people like to read books before they go to sleep
72 lab_193 Friends can cheer you up when you are having a bad day
answerKey itemNum attempted correct pCorr pCorrStd sum avgRT
2 FALSE 3 41 37 0.9024390 -0.03865587 56.47353 1.411838
3 TRUE 7 54 50 0.9259259 0.29250144 129.53265 2.444012
4 TRUE 10 34 32 0.9411765 0.50752893 44.29897 1.384343
5 FALSE 11 46 43 0.9347826 0.41737766 141.54039 3.145342
6 FALSE 13 50 47 0.9400000 0.49094110 100.29683 2.046874
7 TRUE 14 46 42 0.9130435 0.11086332 116.87339 2.597187
8 FALSE 15 34 32 0.9411765 0.50752893 58.45122 1.719154
9 TRUE 18 43 39 0.9069767 0.02532443 87.28487 2.128899
10 TRUE 21 46 42 0.9130435 0.11086332 77.55469 1.723438
11 TRUE 22 35 32 0.9142857 0.12837842 76.54983 2.251466
12 TRUE 27 34 31 0.9117647 0.09283306 100.60881 3.048752
13 TRUE 28 33 30 0.9090909 0.05513344 60.19951 1.824228
14 FALSE 30 37 34 0.9189189 0.19370503 86.52711 2.403531
15 TRUE 31 41 38 0.9268293 0.30523826 83.98658 2.153502
16 TRUE 36 44 40 0.9090909 0.05513344 128.42074 2.986529
17 FALSE 37 33 31 0.9393939 0.48239585 93.09233 2.820980
18 FALSE 38 38 35 0.9210526 0.22378965 86.69107 2.281344
19 TRUE 39 44 40 0.9090909 0.05513344 126.34444 3.008201
20 FALSE 45 42 38 0.9047619 -0.00590405 106.11772 2.652943
21 FALSE 49 38 35 0.9210526 0.22378965 110.77468 2.915123
22 FALSE 50 41 37 0.9024390 -0.03865587 107.19094 2.679773
23 TRUE 51 35 32 0.9142857 0.12837842 79.38833 2.268238
24 FALSE 53 42 39 0.9285714 0.32980213 73.74313 1.798613
25 FALSE 58 42 39 0.9285714 0.32980213 135.62898 3.308024
26 TRUE 60 36 33 0.9166667 0.16194904 81.75420 2.270950
28 FALSE 64 34 31 0.9117647 0.09283306 130.60357 3.957684
29 FALSE 66 33 30 0.9090909 0.05513344 74.02196 2.387805
30 FALSE 70 34 32 0.9411765 0.50752893 103.98862 3.058489
32 FALSE 81 40 38 0.9500000 0.63193769 108.47540 2.711885
33 TRUE 88 37 34 0.9189189 0.19370503 74.58953 2.071932
34 FALSE 91 47 43 0.9148936 0.13694964 164.55758 3.501225
35 FALSE 95 47 44 0.9361702 0.43694240 117.58834 2.501880
36 FALSE 98 35 33 0.9428571 0.53122584 96.19204 2.829178
37 FALSE 100 36 34 0.9444444 0.55360625 136.59557 3.794321
38 FALSE 101 42 39 0.9285714 0.32980213 150.71802 3.767951
40 FALSE 105 52 48 0.9230769 0.25233147 185.66851 3.640559
41 TRUE 107 42 38 0.9047619 -0.00590405 150.20310 3.663490
43 FALSE 110 38 36 0.9473684 0.59483333 142.59378 3.853886
44 TRUE 114 45 42 0.9333333 0.39694337 106.92999 2.430227
45 FALSE 119 35 33 0.9428571 0.53122584 91.81476 2.782266
46 TRUE 120 39 36 0.9230769 0.25233147 128.24418 3.288312
47 TRUE 121 34 31 0.9117647 0.09283306 119.06712 3.501974
48 TRUE 123 49 45 0.9183673 0.18592805 148.56686 3.031977
49 FALSE 124 40 37 0.9250000 0.27944620 135.08062 3.463606
50 TRUE 127 33 31 0.9393939 0.48239585 88.90841 2.778388
51 TRUE 128 47 44 0.9361702 0.43694240 184.63177 3.928336
53 FALSE 134 37 35 0.9459459 0.57477691 117.87427 3.185791
54 TRUE 136 48 44 0.9166667 0.16194904 234.40883 4.883517
56 FALSE 144 42 39 0.9285714 0.32980213 143.57695 3.589424
57 FALSE 145 34 31 0.9117647 0.09283306 124.27937 3.766042
58 TRUE 146 34 31 0.9117647 0.09283306 125.22210 3.794609
59 TRUE 153 33 31 0.9393939 0.48239585 112.60909 3.519034
60 TRUE 154 32 30 0.9375000 0.45569195 140.25530 4.524365
61 FALSE 157 46 43 0.9347826 0.41737766 169.29087 3.936997
62 TRUE 165 39 36 0.9230769 0.25233147 161.19087 4.241865
63 FALSE 166 32 30 0.9375000 0.45569195 125.63388 4.052706
64 FALSE 167 37 35 0.9459459 0.57477691 98.14500 2.804143
65 TRUE 169 49 45 0.9183673 0.18592805 132.50265 2.760472
67 TRUE 176 43 40 0.9302326 0.35322349 138.08943 3.211382
68 TRUE 179 33 31 0.9393939 0.48239585 124.76787 3.898996
69 FALSE 182 44 40 0.9090909 0.05513344 178.93396 4.066681
70 TRUE 183 43 40 0.9302326 0.35322349 145.54502 3.465358
71 TRUE 184 49 45 0.9183673 0.18592805 177.13930 3.690402
72 TRUE 193 42 38 0.9047619 -0.00590405 117.03945 2.786653
avgRTStd numCharacter difficulty distance weights subclass
2 -1.79250328 12 Medium 0.6233724 1 31
3 -0.75655902 12 Medium 0.5656060 1 10
4 -1.82009928 14 Medium 0.5270103 1 12
5 -0.05266739 17 Medium 0.5432581 1 11
6 -1.15514793 19 Medium 0.5300052 1 5
7 -0.60282510 24 Medium 0.5976186 1 28
8 -1.48406552 15 Medium 0.5270103 1 29
9 -1.07282289 24 Medium 0.6124280 1 32
10 -1.47976573 17 Medium 0.5976186 1 1
11 -0.94980896 19 Medium 0.5945631 1 2
12 -0.14961015 18 Medium 0.6007560 1 3
13 -1.37860749 14 Medium 0.6072892 1 4
14 -0.79718809 18 Medium 0.5831041 1 16
15 -1.04813035 19 Medium 0.5633372 1 5
16 -0.21206070 22 Medium 0.6072892 1 6
17 -0.37821431 22 Medium 0.5315473 1 10
18 -0.91982146 24 Medium 0.5777957 1 1
19 -0.19030930 24 Medium 0.6072892 1 7
20 -0.54686498 22 Medium 0.6177849 1 13
21 -0.28372710 21 Medium 0.5777957 1 28
22 -0.51993660 23 Medium 0.6233724 1 21
23 -0.93297543 25 Medium 0.5945631 1 8
24 -1.40431574 19 Medium 0.5589543 1 25
25 0.11060888 33 Medium 0.5589543 1 17
26 -0.93025338 21 Medium 0.5886865 1 9
28 0.76264208 23 Medium 0.6007560 1 7
29 -0.81297151 24 Medium 0.6072892 1 4
30 -0.13983788 26 Medium 0.5270103 1 26
32 -0.48770755 27 Medium 0.5045013 1 12
33 -1.12999885 24 Medium 0.5831041 1 11
34 0.30451595 30 Medium 0.5930652 1 3
35 -0.69848013 32 Medium 0.5397386 1 15
36 -0.36998656 23 Medium 0.5227286 1 19
37 0.59868278 39 Medium 0.5186816 1 23
38 0.57221569 36 Medium 0.5589543 1 20
40 0.44435863 31 Medium 0.5727429 1 2
41 0.46737360 28 Medium 0.6177849 1 13
43 0.65846492 35 Medium 0.5112206 1 27
44 -0.77039439 25 Medium 0.5469294 1 14
45 -0.41706997 34 Medium 0.5227286 1 14
46 0.09082529 34 Medium 0.5727429 1 15
47 0.30526759 34 Medium 0.6007560 1 16
48 -0.16644666 39 Medium 0.5844732 1 17
49 0.26675882 42 Medium 0.5679286 1 8
50 -0.42096193 24 Medium 0.5315473 1 18
51 0.73318658 29 Medium 0.5397386 1 19
53 -0.01207044 34 Medium 0.5148511 1 22
54 1.69185734 28 Medium 0.5886865 1 20
56 0.39303667 30 Medium 0.5589543 1 9
57 0.57029960 36 Medium 0.6007560 1 6
58 0.59897154 30 Medium 0.6007560 1 21
59 0.32239001 37 Medium 0.5315473 1 22
60 1.33139271 44 Medium 0.5363620 1 23
61 0.74187955 41 Medium 0.5432581 1 30
62 1.04786111 48 Medium 0.5727429 1 24
63 0.85801106 44 Medium 0.5363620 1 24
64 -0.39511266 36 Medium 0.5148511 1 18
65 -0.43894331 35 Medium 0.5844732 1 25
67 0.01361405 37 Medium 0.5547667 1 26
68 0.70373970 29 Medium 0.5315473 1 27
69 0.87203733 45 Medium 0.6072892 1 32
70 0.26851733 45 Medium 0.5547667 1 29
71 0.49438387 44 Medium 0.5844732 1 30
72 -0.41266609 43 Medium 0.6177849 1 31After matching the hard sentences, 26 sentences (13 true and 13 false) were included in the final set of sentences.
df.hard.excluded sentenceId sentence answerKey
1 lab_20 Fire gives you frostbite FALSE
2 lab_26 All girls wear dresses FALSE
3 lab_59 Camels enjoy wearing flip-flops FALSE
4 lab_67 Stones are kept in the fridge FALSE
5 lab_97 Every fish lives in a fishbowl FALSE
6 lab_108 The sun is smaller than the Earth FALSE
7 lab_125 Every door is a sliding glass door FALSE
8 lab_130 Leaves on trees turn green when they die FALSE
9 lab_173 Most adults walk with their hands instead of their feet FALSE
itemNum attempted correct pCorr pCorrStd sum avgRT avgRTStd
1 20 42 37 0.8809524 -0.3416102 94.69809 2.367452 -0.83339861
2 26 44 39 0.8863636 -0.2653134 99.65924 2.264983 -0.93624256
3 59 36 31 0.8611111 -0.6213654 85.44953 2.513221 -0.68709690
4 67 37 32 0.8648649 -0.5684387 71.62725 2.046493 -1.15553045
5 97 36 32 0.8888889 -0.2297082 126.69766 3.519379 0.32273649
6 108 37 33 0.8918919 -0.1873669 82.42029 2.354865 -0.84603155
7 125 37 33 0.8918919 -0.1873669 117.72194 3.270054 0.07250017
8 130 46 41 0.8913043 -0.1956510 188.94052 4.198678 1.00451670
9 173 37 32 0.8648649 -0.5684387 141.11462 3.919851 0.72467064
numCharacter difficulty
1 21 Hard
2 19 Hard
3 28 Hard
4 24 Hard
5 25 Hard
6 27 Hard
7 28 Hard
8 33 Hard
9 46 Harddf.lab.items.hard.final sentenceId sentence
1 lab_9 Wheels are round
2 lab_17 Doors are always open
5 lab_34 A raven is a bird
6 lab_41 Dish washers make plates dirty
7 lab_48 A tricycle has two wheels
8 lab_52 Trampolines help you bounce higher
10 lab_68 A cactus grows under the sea
11 lab_76 There are many types of lizards
12 lab_80 Television shows are longer than movies
13 lab_86 Ice helps to keep things cold
14 lab_92 A gardener works at a bakery
15 lab_96 Snow always falls in the summer
17 lab_102 Halloween is a holiday in February
19 lab_111 A mouse is a rodent that squeaks
20 lab_113 Stones can be found near a river
21 lab_117 Hats should be worn on your elbow
24 lab_131 Every child holds pencils with their left hand
25 lab_133 Most babies know how to drive a car
26 lab_139 Some people get to work using a spaceship
27 lab_142 Police officers drive school busses while on duty
28 lab_148 Friends can do many things together after school
29 lab_149 Another name for a present is a gift
30 lab_152 The weather in the winter can be unpredictable
31 lab_156 People can use credit cards to pay in stores
33 lab_185 Trashcans can become smelly if trash is left inside too long
34 lab_187 Most people play board games with at least one other person
answerKey itemNum attempted correct pCorr pCorrStd sum avgRT
1 TRUE 9 40 35 0.8750000 -0.4255368 66.24548 1.698602
2 FALSE 17 36 31 0.8611111 -0.6213654 84.30579 2.408737
5 TRUE 34 46 40 0.8695652 -0.5021654 84.24562 1.831426
6 FALSE 41 41 36 0.8780488 -0.3825500 133.94117 3.266858
7 FALSE 48 39 34 0.8717949 -0.4707280 116.55780 2.988662
8 TRUE 52 35 31 0.8857143 -0.2744690 89.75446 2.639837
10 FALSE 68 47 41 0.8723404 -0.4630359 108.47005 2.410446
11 TRUE 76 38 34 0.8947368 -0.1472540 110.11218 2.897689
12 FALSE 80 41 36 0.8780488 -0.3825500 147.65833 3.691458
13 TRUE 86 41 36 0.8780488 -0.3825500 101.78939 2.482668
14 FALSE 92 38 34 0.8947368 -0.1472540 94.15842 2.544822
15 FALSE 96 38 34 0.8947368 -0.1472540 148.46899 4.012675
17 FALSE 102 42 37 0.8809524 -0.3416102 117.20619 2.790623
19 TRUE 111 42 37 0.8809524 -0.3416102 199.60910 4.752598
20 TRUE 113 38 34 0.8947368 -0.1472540 154.91296 4.186837
21 FALSE 117 35 31 0.8857143 -0.2744690 109.86183 3.329146
24 FALSE 131 33 29 0.8787879 -0.3721290 105.47607 3.196244
25 FALSE 133 42 37 0.8809524 -0.3416102 133.25655 3.250160
26 FALSE 139 27 23 0.8518519 -0.7519178 134.02384 4.963846
27 FALSE 142 39 35 0.8974359 -0.1091983 196.32806 5.306164
28 TRUE 148 39 35 0.8974359 -0.1091983 159.18083 4.081560
29 TRUE 149 38 33 0.8684211 -0.5182977 210.40058 5.536857
30 TRUE 152 40 35 0.8750000 -0.4255368 213.40882 5.335221
31 TRUE 156 32 28 0.8750000 -0.4255368 95.05799 3.066387
33 TRUE 185 34 29 0.8529412 -0.7365587 127.02520 3.969537
34 TRUE 187 35 30 0.8571429 -0.6773164 171.10255 5.032428
avgRTStd numCharacter difficulty distance weights subclass
1 -1.504691943 14 Hard 0.4001698 1 1
2 -0.791963193 18 Hard 0.4663906 1 11
5 -1.371382512 13 Hard 0.4257874 1 12
6 0.069292288 26 Hard 0.3860251 1 8
7 -0.209920087 21 Hard 0.4152211 1 7
8 -0.560018946 30 Hard 0.3513447 1 13
10 -0.790248253 23 Hard 0.4126471 1 12
11 -0.301225127 26 Hard 0.3124676 1 2
12 0.495443891 34 Hard 0.3860251 1 1
13 -0.717762058 24 Hard 0.3860251 1 3
14 -0.655380914 23 Hard 0.3124676 1 2
15 0.817834431 26 Hard 0.3124676 1 5
17 -0.408681593 29 Hard 0.3727302 1 4
19 1.560459528 26 Hard 0.3727302 1 4
20 0.992631846 26 Hard 0.3124676 1 5
21 0.131808505 27 Hard 0.3513447 1 13
24 -0.001578946 39 Hard 0.3826239 1 9
25 0.052533487 28 Hard 0.3727302 1 3
26 1.772479511 34 Hard 0.5113574 1 10
27 2.116047832 42 Hard 0.3013030 1 6
28 0.886970399 41 Hard 0.3013030 1 6
29 2.347584124 29 Hard 0.4312367 1 7
30 2.145210816 39 Hard 0.4001698 1 8
31 -0.131910927 36 Hard 0.4001698 1 9
33 0.774538946 50 Hard 0.5060627 1 10
34 1.841311904 49 Hard 0.4856379 1 11We are left with a final stimuli bank of 130 sentences, consisting of 40 easy, 64 medium, and 26 hard sentences.
df.lab.items.final <- df.lab.items.easy.final %>%
rbind(df.lab.items.medium.final) %>%
rbind(df.lab.items.hard.final) df.lab.items.final sentenceId
1 lab_0
2 lab_1
3 lab_4
4 lab_6
5 lab_8
6 lab_12
7 lab_16
8 lab_25
9 lab_32
10 lab_33
11 lab_35
12 lab_40
13 lab_44
16 lab_55
17 lab_61
18 lab_63
19 lab_69
20 lab_71
21 lab_73
22 lab_74
23 lab_75
24 lab_77
28 lab_85
29 lab_87
30 lab_89
31 lab_90
32 lab_93
33 lab_99
34 lab_106
35 lab_115
36 lab_116
37 lab_122
40 lab_140
42 lab_162
43 lab_164
44 lab_170
46 lab_178
47 lab_180
49 lab_186
50 lab_191
27 lab_3
39 lab_7
410 lab_10
52 lab_11
66 lab_13
73 lab_14
81 lab_15
91 lab_18
101 lab_21
111 lab_22
121 lab_27
131 lab_28
14 lab_30
15 lab_31
161 lab_36
171 lab_37
181 lab_38
191 lab_39
201 lab_45
211 lab_49
221 lab_50
231 lab_51
241 lab_53
25 lab_58
26 lab_60
281 lab_64
291 lab_66
301 lab_70
321 lab_81
331 lab_88
341 lab_91
351 lab_95
361 lab_98
371 lab_100
38 lab_101
401 lab_105
41 lab_107
431 lab_110
441 lab_114
45 lab_119
461 lab_120
471 lab_121
48 lab_123
491 lab_124
501 lab_127
51 lab_128
53 lab_134
54 lab_136
56 lab_144
57 lab_145
58 lab_146
59 lab_153
60 lab_154
61 lab_157
62 lab_165
63 lab_166
64 lab_167
65 lab_169
67 lab_176
68 lab_179
69 lab_182
70 lab_183
71 lab_184
72 lab_193
110 lab_9
210 lab_17
55 lab_34
610 lab_41
74 lab_48
82 lab_52
102 lab_68
112 lab_76
122 lab_80
132 lab_86
141 lab_92
151 lab_96
172 lab_102
192 lab_111
202 lab_113
212 lab_117
242 lab_131
251 lab_133
261 lab_139
271 lab_142
282 lab_148
292 lab_149
302 lab_152
311 lab_156
332 lab_185
342 lab_187
sentence
1 Chairs are alive
2 Fish have thumbs
3 Cars have wheels
4 Trees have branches
5 Sandwiches are food
6 Soda is always brown
7 Children can be sad
8 All cars are yellow
9 Triangles have five sides
10 A monkey has a face
11 People like to eat mud
12 Children enjoy playing with toys
13 Computers need electricity to work
16 Some houses have back yards
17 Some kids take music lessons
18 Spaceships are flown by cowboys
19 A pillow can be very soft
20 Some worms ride pumpkins to school
21 Sweaters can be made of coal
22 Chefs cook meals in their beds
23 Dogs always walk on two legs
24 Trash is thrown away in bathtubs
28 Eyeglasses can help you see better
29 Wolves are a type of monkey
30 It is usually dark at night
31 A blanket can be very soft
32 Towels are often made of glass
33 Most humans listen with their feet
34 Some children are afraid of the dark
35 People can buy tomatoes at the market
36 All children are afraid of the sun
37 Orange juice is always made of grapes
40 The ocean is home to many aquatic animals
42 Trash cans are where you store extra notebook paper
43 Basketball is a game that you play with a fork
44 Some people like to run for exercise in the morning
46 Soccer is a game you play with only your hands
47 A teacher is a person that works at a school
49 Grandparents can bring presents when they come to visit their family
50 When you break a bone you normally have to wear a cast
27 Bears can talk
39 Mice have ears
410 The sun is bright
52 Whales have two legs
66 All flowers are yellow
73 People sometimes drink milk
81 Cows sing at night
91 Elephants are large animals
101 Apples grow on trees
111 Kittens are young cats
121 Some shoes have laces
131 Adults can be sad
14 Moms like eating hats
15 Many babies drink milk
161 Some kids love red crayons
171 Lizards like to cook pasta
181 Snails are usually very fast
191 Pets can keep people company
201 Blankets are made of metal
211 Every magician is a fairy
221 Blankets help keep you cold
231 Many nurses work in hospitals
241 People like to eat dust
25 Tigers sometimes swing at playgrounds
26 Some girls play the piano
281 Cloudy days are always warm
291 People drive cars in the sink
301 A curtain is something you wear
321 Eating only ice cream is healthy
331 People drive cars on the road
341 Some slugs ride elephants to school
351 Puppies carry backpacks on their back
361 Most houses are made of foam
371 Many people paint pictures with screwdrivers
38 Lollipops are always made from vegetables
401 Most houses have grass on the ceiling
41 Pockets can be used to keep change
431 People brush their teeth with mayonnaise
441 A cactus may grow in the desert
45 Boys usually wash their hair with yogurt
461 Girls can listen to music with earphones
471 Climbing up a mountain can be exhausting
48 Adults are usually taller than their children
491 Dentists always give children candy and lemonade
501 A picture can hang on the wall
51 People can use cash to pay in stores
53 The United States of America is in Africa
54 Towels are used to make things dry
56 Most girls keep their books in a pond
57 Every basketball player is on a school team
58 Stores are where you go to buy things
59 People can wear socks to keep their feet warm
60 Air conditioners can lower the temperature of a room
61 Medicine is supposed to make you stay sick longer
62 Umbrellas are useful for keeping people dry when it rains
63 Spiders like to drink orange juice out of coffee mugs
64 Baseball is a game that you play with a knife
65 When a traffic light is red you need to stop
67 Most dogs wear leashes when they go for a walk
68 It is faster to run than it is to walk
69 Students never get in trouble when they cheat on tests
70 A doctor can work in a hospital that serves many people
71 Some people like to read books before they go to sleep
72 Friends can cheer you up when you are having a bad day
110 Wheels are round
210 Doors are always open
55 A raven is a bird
610 Dish washers make plates dirty
74 A tricycle has two wheels
82 Trampolines help you bounce higher
102 A cactus grows under the sea
112 There are many types of lizards
122 Television shows are longer than movies
132 Ice helps to keep things cold
141 A gardener works at a bakery
151 Snow always falls in the summer
172 Halloween is a holiday in February
192 A mouse is a rodent that squeaks
202 Stones can be found near a river
212 Hats should be worn on your elbow
242 Every child holds pencils with their left hand
251 Most babies know how to drive a car
261 Some people get to work using a spaceship
271 Police officers drive school busses while on duty
282 Friends can do many things together after school
292 Another name for a present is a gift
302 The weather in the winter can be unpredictable
311 People can use credit cards to pay in stores
332 Trashcans can become smelly if trash is left inside too long
342 Most people play board games with at least one other person
answerKey itemNum attempted correct pCorr pCorrStd sum
1 FALSE 0 44 42 0.9545455 0.69602706 91.49515
2 FALSE 1 45 45 1.0000000 1.33692067 85.00037
3 TRUE 4 42 41 0.9761905 1.00121449 66.08075
4 TRUE 6 41 40 0.9756098 0.99302654 67.65853
5 TRUE 8 48 47 0.9791667 1.04317777 73.21354
6 FALSE 12 43 41 0.9534884 0.68112255 93.69506
7 TRUE 16 41 40 0.9756098 0.99302654 66.20243
8 FALSE 25 35 35 1.0000000 1.33692067 62.23937
9 FALSE 32 40 40 1.0000000 1.33692067 93.43161
10 TRUE 33 32 32 1.0000000 1.33692067 51.59473
11 FALSE 35 47 45 0.9574468 0.73693516 89.03229
12 TRUE 40 37 37 1.0000000 1.33692067 89.03335
13 TRUE 44 29 29 1.0000000 1.33692067 75.53671
16 TRUE 55 43 43 1.0000000 1.33692067 89.82713
17 TRUE 61 39 39 1.0000000 1.33692067 88.94155
18 FALSE 63 44 44 1.0000000 1.33692067 127.36084
19 TRUE 69 48 47 0.9791667 1.04317777 105.97524
20 FALSE 71 34 33 0.9705882 0.92222480 104.39870
21 FALSE 73 41 39 0.9512195 0.64913240 125.96161
22 FALSE 74 41 40 0.9756098 0.99302654 118.94683
23 FALSE 75 42 41 0.9761905 1.00121449 123.97459
24 FALSE 77 45 43 0.9555556 0.71026914 136.87046
28 TRUE 85 38 37 0.9736842 0.96587700 108.14471
29 FALSE 87 41 39 0.9512195 0.64913240 76.64538
30 TRUE 89 38 38 1.0000000 1.33692067 85.07677
31 TRUE 90 34 34 1.0000000 1.33692067 105.06983
32 FALSE 93 43 43 1.0000000 1.33692067 93.89001
33 FALSE 99 45 44 0.9777778 1.02359491 116.48522
34 TRUE 106 37 37 1.0000000 1.33692067 82.42713
35 TRUE 115 37 37 1.0000000 1.33692067 115.53913
36 FALSE 116 40 39 0.9750000 0.98442918 96.30437
37 FALSE 122 34 34 1.0000000 1.33692067 78.62520
40 TRUE 140 38 37 0.9736842 0.96587700 128.94337
42 FALSE 162 35 34 0.9714286 0.93407326 197.23792
43 FALSE 164 45 45 1.0000000 1.33692067 135.32318
44 TRUE 170 51 50 0.9803922 1.06045676 172.20617
46 FALSE 178 35 34 0.9714286 0.93407326 122.21326
47 TRUE 180 38 37 0.9736842 0.96587700 93.46986
49 TRUE 186 38 37 0.9736842 0.96587700 165.01177
50 TRUE 191 41 40 0.9756098 0.99302654 166.95486
27 FALSE 3 41 37 0.9024390 -0.03865587 56.47353
39 TRUE 7 54 50 0.9259259 0.29250144 129.53265
410 TRUE 10 34 32 0.9411765 0.50752893 44.29897
52 FALSE 11 46 43 0.9347826 0.41737766 141.54039
66 FALSE 13 50 47 0.9400000 0.49094110 100.29683
73 TRUE 14 46 42 0.9130435 0.11086332 116.87339
81 FALSE 15 34 32 0.9411765 0.50752893 58.45122
91 TRUE 18 43 39 0.9069767 0.02532443 87.28487
101 TRUE 21 46 42 0.9130435 0.11086332 77.55469
111 TRUE 22 35 32 0.9142857 0.12837842 76.54983
121 TRUE 27 34 31 0.9117647 0.09283306 100.60881
131 TRUE 28 33 30 0.9090909 0.05513344 60.19951
14 FALSE 30 37 34 0.9189189 0.19370503 86.52711
15 TRUE 31 41 38 0.9268293 0.30523826 83.98658
161 TRUE 36 44 40 0.9090909 0.05513344 128.42074
171 FALSE 37 33 31 0.9393939 0.48239585 93.09233
181 FALSE 38 38 35 0.9210526 0.22378965 86.69107
191 TRUE 39 44 40 0.9090909 0.05513344 126.34444
201 FALSE 45 42 38 0.9047619 -0.00590405 106.11772
211 FALSE 49 38 35 0.9210526 0.22378965 110.77468
221 FALSE 50 41 37 0.9024390 -0.03865587 107.19094
231 TRUE 51 35 32 0.9142857 0.12837842 79.38833
241 FALSE 53 42 39 0.9285714 0.32980213 73.74313
25 FALSE 58 42 39 0.9285714 0.32980213 135.62898
26 TRUE 60 36 33 0.9166667 0.16194904 81.75420
281 FALSE 64 34 31 0.9117647 0.09283306 130.60357
291 FALSE 66 33 30 0.9090909 0.05513344 74.02196
301 FALSE 70 34 32 0.9411765 0.50752893 103.98862
321 FALSE 81 40 38 0.9500000 0.63193769 108.47540
331 TRUE 88 37 34 0.9189189 0.19370503 74.58953
341 FALSE 91 47 43 0.9148936 0.13694964 164.55758
351 FALSE 95 47 44 0.9361702 0.43694240 117.58834
361 FALSE 98 35 33 0.9428571 0.53122584 96.19204
371 FALSE 100 36 34 0.9444444 0.55360625 136.59557
38 FALSE 101 42 39 0.9285714 0.32980213 150.71802
401 FALSE 105 52 48 0.9230769 0.25233147 185.66851
41 TRUE 107 42 38 0.9047619 -0.00590405 150.20310
431 FALSE 110 38 36 0.9473684 0.59483333 142.59378
441 TRUE 114 45 42 0.9333333 0.39694337 106.92999
45 FALSE 119 35 33 0.9428571 0.53122584 91.81476
461 TRUE 120 39 36 0.9230769 0.25233147 128.24418
471 TRUE 121 34 31 0.9117647 0.09283306 119.06712
48 TRUE 123 49 45 0.9183673 0.18592805 148.56686
491 FALSE 124 40 37 0.9250000 0.27944620 135.08062
501 TRUE 127 33 31 0.9393939 0.48239585 88.90841
51 TRUE 128 47 44 0.9361702 0.43694240 184.63177
53 FALSE 134 37 35 0.9459459 0.57477691 117.87427
54 TRUE 136 48 44 0.9166667 0.16194904 234.40883
56 FALSE 144 42 39 0.9285714 0.32980213 143.57695
57 FALSE 145 34 31 0.9117647 0.09283306 124.27937
58 TRUE 146 34 31 0.9117647 0.09283306 125.22210
59 TRUE 153 33 31 0.9393939 0.48239585 112.60909
60 TRUE 154 32 30 0.9375000 0.45569195 140.25530
61 FALSE 157 46 43 0.9347826 0.41737766 169.29087
62 TRUE 165 39 36 0.9230769 0.25233147 161.19087
63 FALSE 166 32 30 0.9375000 0.45569195 125.63388
64 FALSE 167 37 35 0.9459459 0.57477691 98.14500
65 TRUE 169 49 45 0.9183673 0.18592805 132.50265
67 TRUE 176 43 40 0.9302326 0.35322349 138.08943
68 TRUE 179 33 31 0.9393939 0.48239585 124.76787
69 FALSE 182 44 40 0.9090909 0.05513344 178.93396
70 TRUE 183 43 40 0.9302326 0.35322349 145.54502
71 TRUE 184 49 45 0.9183673 0.18592805 177.13930
72 TRUE 193 42 38 0.9047619 -0.00590405 117.03945
110 TRUE 9 40 35 0.8750000 -0.42553678 66.24548
210 FALSE 17 36 31 0.8611111 -0.62136538 84.30579
55 TRUE 34 46 40 0.8695652 -0.50216536 84.24562
610 FALSE 41 41 36 0.8780488 -0.38255001 133.94117
74 FALSE 48 39 34 0.8717949 -0.47072799 116.55780
82 TRUE 52 35 31 0.8857143 -0.27446899 89.75446
102 FALSE 68 47 41 0.8723404 -0.46303587 108.47005
112 TRUE 76 38 34 0.8947368 -0.14725402 110.11218
122 FALSE 80 41 36 0.8780488 -0.38255001 147.65833
132 TRUE 86 41 36 0.8780488 -0.38255001 101.78939
141 FALSE 92 38 34 0.8947368 -0.14725402 94.15842
151 FALSE 96 38 34 0.8947368 -0.14725402 148.46899
172 FALSE 102 42 37 0.8809524 -0.34161023 117.20619
192 TRUE 111 42 37 0.8809524 -0.34161023 199.60910
202 TRUE 113 38 34 0.8947368 -0.14725402 154.91296
212 FALSE 117 35 31 0.8857143 -0.27446899 109.86183
242 FALSE 131 33 29 0.8787879 -0.37212897 105.47607
251 FALSE 133 42 37 0.8809524 -0.34161023 133.25655
261 FALSE 139 27 23 0.8518519 -0.75191779 134.02384
271 FALSE 142 39 35 0.8974359 -0.10919826 196.32806
282 TRUE 148 39 35 0.8974359 -0.10919826 159.18083
292 TRUE 149 38 33 0.8684211 -0.51829769 210.40058
302 TRUE 152 40 35 0.8750000 -0.42553678 213.40882
311 TRUE 156 32 28 0.8750000 -0.42553678 95.05799
332 TRUE 185 34 29 0.8529412 -0.73655868 127.02520
342 TRUE 187 35 30 0.8571429 -0.67731641 171.10255
avgRT avgRTStd numCharacter difficulty distance weights subclass
1 2.079435 -1.122467853 14 Easy 0.5585887 1 1_1_7
2 1.888897 -1.313701989 14 Easy 0.6372482 1 1_1_3
3 1.611726 -1.591885867 14 Easy 0.5966747 1 1_1_8
4 1.650208 -1.553262854 17 Easy 0.5956658 1 1_1_13
5 1.525282 -1.678645270 17 Easy 0.6018322 1 1_1_18
6 2.178955 -1.022584427 17 Easy 0.5567070 1 1_1_14
7 1.614694 -1.588907098 16 Easy 0.5956658 1 1_1_20
8 1.778268 -1.424735539 16 Easy 0.6372482 1 1_1_2
9 2.335790 -0.865176269 22 Easy 0.6372482 1 1_1_1
10 1.612335 -1.591273937 15 Easy 0.6372482 1 1_1_1
11 1.894304 -1.308275088 18 Easy 0.5637444 1 1_1_20
12 2.406307 -0.794402170 28 Easy 0.6372482 1 1_1_2
13 2.604714 -0.595270021 30 Easy 0.6372482 1 1_1_3
16 2.089003 -1.112864992 23 Easy 0.6372482 1 1_1_4
17 2.280552 -0.920615863 24 Easy 0.6372482 1 1_1_5
18 2.894565 -0.304360810 27 Easy 0.6372482 1 1_1_4
19 2.207817 -0.993616599 20 Easy 0.6018322 1 1_1_6
20 3.070550 -0.127732493 29 Easy 0.5869091 1 1_1_13
21 3.072235 -0.126041827 23 Easy 0.5526629 1 1_1_17
22 2.973671 -0.224965511 25 Easy 0.5956658 1 1_1_15
23 2.951776 -0.246940533 23 Easy 0.5966747 1 1_1_12
24 3.110692 -0.087443697 27 Easy 0.5603851 1 1_1_19
28 2.922830 -0.275991968 29 Easy 0.5923149 1 1_1_7
29 1.965266 -1.237053993 22 Easy 0.5526629 1 1_1_16
30 2.238862 -0.962458221 22 Easy 0.6372482 1 1_1_9
31 3.090289 -0.107921270 21 Easy 0.6372482 1 1_1_10
32 2.183489 -1.018034262 25 Easy 0.6372482 1 1_1_5
33 2.588561 -0.611482611 29 Easy 0.5994281 1 1_1_11
34 2.227760 -0.973601010 30 Easy 0.6372482 1 1_1_11
35 3.122679 -0.075412880 31 Easy 0.6372482 1 1_1_12
36 2.407609 -0.793094794 28 Easy 0.5946056 1 1_1_18
37 2.312506 -0.888545545 31 Easy 0.6372482 1 1_1_9
40 3.484956 0.288187249 34 Easy 0.5923149 1 1_1_14
42 5.635369 2.446455755 43 Easy 0.5883785 1 1_1_8
43 3.007182 -0.191332195 37 Easy 0.6372482 1 1_1_10
44 3.376592 0.179427122 42 Easy 0.6039494 1 1_1_15
46 3.491807 0.295063760 37 Easy 0.5883785 1 1_1_6
47 2.459733 -0.740780606 35 Easy 0.5923149 1 1_1_16
49 4.342415 1.148778409 58 Easy 0.5923149 1 1_1_17
50 4.072070 0.877445867 43 Easy 0.5956658 1 1_1_19
27 1.411838 -1.792503276 12 Medium 0.6233724 1 1_2_31
39 2.444012 -0.756559022 12 Medium 0.5656060 1 1_2_10
410 1.384343 -1.820099278 14 Medium 0.5270103 1 1_2_12
52 3.145342 -0.052667395 17 Medium 0.5432581 1 1_2_11
66 2.046874 -1.155147930 19 Medium 0.5300052 1 1_2_5
73 2.597187 -0.602825101 24 Medium 0.5976186 1 1_2_28
81 1.719154 -1.484065521 15 Medium 0.5270103 1 1_2_29
91 2.128899 -1.072822887 24 Medium 0.6124280 1 1_2_32
101 1.723438 -1.479765734 17 Medium 0.5976186 1 1_2_1
111 2.251466 -0.949808963 19 Medium 0.5945631 1 1_2_2
121 3.048752 -0.149610150 18 Medium 0.6007560 1 1_2_3
131 1.824228 -1.378607494 14 Medium 0.6072892 1 1_2_4
14 2.403531 -0.797188095 18 Medium 0.5831041 1 1_2_16
15 2.153502 -1.048130354 19 Medium 0.5633372 1 1_2_5
161 2.986529 -0.212060696 22 Medium 0.6072892 1 1_2_6
171 2.820980 -0.378214307 22 Medium 0.5315473 1 1_2_10
181 2.281344 -0.919821460 24 Medium 0.5777957 1 1_2_1
191 3.008201 -0.190309297 24 Medium 0.6072892 1 1_2_7
201 2.652943 -0.546864982 22 Medium 0.6177849 1 1_2_13
211 2.915123 -0.283727099 21 Medium 0.5777957 1 1_2_28
221 2.679773 -0.519936598 23 Medium 0.6233724 1 1_2_21
231 2.268238 -0.932975426 25 Medium 0.5945631 1 1_2_8
241 1.798613 -1.404315741 19 Medium 0.5589543 1 1_2_25
25 3.308024 0.110608885 33 Medium 0.5589543 1 1_2_17
26 2.270950 -0.930253376 21 Medium 0.5886865 1 1_2_9
281 3.957684 0.762642085 23 Medium 0.6007560 1 1_2_7
291 2.387805 -0.812971513 24 Medium 0.6072892 1 1_2_4
301 3.058489 -0.139837876 26 Medium 0.5270103 1 1_2_26
321 2.711885 -0.487707546 27 Medium 0.5045013 1 1_2_12
331 2.071932 -1.129998849 24 Medium 0.5831041 1 1_2_11
341 3.501225 0.304515946 30 Medium 0.5930652 1 1_2_3
351 2.501880 -0.698480125 32 Medium 0.5397386 1 1_2_15
361 2.829178 -0.369986562 23 Medium 0.5227286 1 1_2_19
371 3.794321 0.598682782 39 Medium 0.5186816 1 1_2_23
38 3.767951 0.572215688 36 Medium 0.5589543 1 1_2_20
401 3.640559 0.444358631 31 Medium 0.5727429 1 1_2_2
41 3.663490 0.467373598 28 Medium 0.6177849 1 1_2_13
431 3.853886 0.658464924 35 Medium 0.5112206 1 1_2_27
441 2.430227 -0.770394389 25 Medium 0.5469294 1 1_2_14
45 2.782266 -0.417069970 34 Medium 0.5227286 1 1_2_14
461 3.288312 0.090825285 34 Medium 0.5727429 1 1_2_15
471 3.501974 0.305267587 34 Medium 0.6007560 1 1_2_16
48 3.031977 -0.166446664 39 Medium 0.5844732 1 1_2_17
491 3.463606 0.266758824 42 Medium 0.5679286 1 1_2_8
501 2.778388 -0.420961929 24 Medium 0.5315473 1 1_2_18
51 3.928336 0.733186576 29 Medium 0.5397386 1 1_2_19
53 3.185791 -0.012070444 34 Medium 0.5148511 1 1_2_22
54 4.883517 1.691857337 28 Medium 0.5886865 1 1_2_20
56 3.589424 0.393036667 30 Medium 0.5589543 1 1_2_9
57 3.766042 0.570299604 36 Medium 0.6007560 1 1_2_6
58 3.794609 0.598971535 30 Medium 0.6007560 1 1_2_21
59 3.519034 0.322390007 37 Medium 0.5315473 1 1_2_22
60 4.524365 1.331392706 44 Medium 0.5363620 1 1_2_23
61 3.936997 0.741879554 41 Medium 0.5432581 1 1_2_30
62 4.241865 1.047861107 48 Medium 0.5727429 1 1_2_24
63 4.052706 0.858011060 44 Medium 0.5363620 1 1_2_24
64 2.804143 -0.395112659 36 Medium 0.5148511 1 1_2_18
65 2.760472 -0.438943312 35 Medium 0.5844732 1 1_2_25
67 3.211382 0.013614050 37 Medium 0.5547667 1 1_2_26
68 3.898996 0.703739700 29 Medium 0.5315473 1 1_2_27
69 4.066681 0.872037327 45 Medium 0.6072892 1 1_2_32
70 3.465358 0.268517331 45 Medium 0.5547667 1 1_2_29
71 3.690402 0.494383867 44 Medium 0.5844732 1 1_2_30
72 2.786653 -0.412666095 43 Medium 0.6177849 1 1_2_31
110 1.698602 -1.504691943 14 Hard 0.4001698 1 2_1
210 2.408737 -0.791963193 18 Hard 0.4663906 1 2_11
55 1.831426 -1.371382512 13 Hard 0.4257874 1 2_12
610 3.266858 0.069292288 26 Hard 0.3860251 1 2_8
74 2.988662 -0.209920087 21 Hard 0.4152211 1 2_7
82 2.639837 -0.560018946 30 Hard 0.3513447 1 2_13
102 2.410446 -0.790248253 23 Hard 0.4126471 1 2_12
112 2.897689 -0.301225127 26 Hard 0.3124676 1 2_2
122 3.691458 0.495443891 34 Hard 0.3860251 1 2_1
132 2.482668 -0.717762058 24 Hard 0.3860251 1 2_3
141 2.544822 -0.655380914 23 Hard 0.3124676 1 2_2
151 4.012675 0.817834431 26 Hard 0.3124676 1 2_5
172 2.790623 -0.408681593 29 Hard 0.3727302 1 2_4
192 4.752598 1.560459528 26 Hard 0.3727302 1 2_4
202 4.186837 0.992631846 26 Hard 0.3124676 1 2_5
212 3.329146 0.131808505 27 Hard 0.3513447 1 2_13
242 3.196244 -0.001578946 39 Hard 0.3826239 1 2_9
251 3.250160 0.052533487 28 Hard 0.3727302 1 2_3
261 4.963846 1.772479511 34 Hard 0.5113574 1 2_10
271 5.306164 2.116047832 42 Hard 0.3013030 1 2_6
282 4.081560 0.886970399 41 Hard 0.3013030 1 2_6
292 5.536857 2.347584124 29 Hard 0.4312367 1 2_7
302 5.335221 2.145210816 39 Hard 0.4001698 1 2_8
311 3.066387 -0.131910927 36 Hard 0.4001698 1 2_9
332 3.969537 0.774538946 50 Hard 0.5060627 1 2_10
342 5.032428 1.841311904 49 Hard 0.4856379 1 2_11set.seed(20)easy.length <- sample(nrow(df.lab.items.easy.final))
easy.final <- df.lab.items.easy.final[easy.length,] %>%
mutate(difficulty = "Easy")medium.length <- sample(nrow(df.lab.items.medium.final))
medium.final <- df.lab.items.medium.final[medium.length,] %>%
mutate(difficulty = "Medium")hard.length <- sample(nrow(df.lab.items.hard.final))
hard.final <- df.lab.items.hard.final[hard.length,] %>%
mutate(difficulty = "Hard")